<!DOCTYPE HTML PUBLIC "-//W3C//DTD HTML 4.01 Transitional//EN" "http://www.w3.org/TR/html4/loose.dtd">
<HTML
><HEAD
><TITLE
>Storing compressed document copies</TITLE
><META
NAME="GENERATOR"
CONTENT="Modular DocBook HTML Stylesheet Version 1.79"><LINK
REL="HOME"
TITLE="DataparkSearch Engine 4.51"
HREF="index.en.html"><LINK
REL="UP"
TITLE="Indexing"
HREF="dpsearch-indexing.en.html"><LINK
REL="PREVIOUS"
TITLE="Using syslog"
HREF="dpsearch-syslog.en.html"><LINK
REL="NEXT"
TITLE="DataparkSearch HTML parser
"
HREF="dpsearch-htmlparser.en.html"><LINK
REL="STYLESHEET"
TYPE="text/css"
HREF="datapark.css"><META
NAME="Description"
CONTENT="DataparkSearch - Full Featured Web site Open Source Search Engine Software over the Internet and Intranet Web Sites Based on SQL Database. It is a Free search software covered by GNU license."><META
NAME="Keywords"
CONTENT="shareware, freeware, download, internet, unix, utilities, search engine, text retrieval, knowledge retrieval, text search, information retrieval, database search, mining, intranet, webserver, index, spider, filesearch, meta, free, open source, full-text, udmsearch, website, find, opensource, search, searching, software, udmsearch, engine, indexing, system, web, ftp, http, cgi, php, SQL, MySQL, database, php3, FreeBSD, Linux, Unix, DataparkSearch, MacOS X, Mac OS X, Windows, 2000, NT, 95, 98, GNU, GPL, url, grabbing"></HEAD
><BODY
CLASS="sect1"
BGCOLOR="#FFFFFF"
TEXT="#000000"
LINK="#0000C4"
VLINK="#1200B2"
ALINK="#C40000"
><!--#include virtual="body-before.html"--><DIV
CLASS="NAVHEADER"
><TABLE
SUMMARY="Header navigation table"
WIDTH="100%"
BORDER="0"
CELLPADDING="0"
CELLSPACING="0"
><TR
><TH
COLSPAN="3"
ALIGN="center"
>DataparkSearch Engine 4.51: Reference manual</TH
></TR
><TR
><TD
WIDTH="10%"
ALIGN="left"
VALIGN="bottom"
><A
HREF="dpsearch-syslog.en.html"
ACCESSKEY="P"
>Prev</A
></TD
><TD
WIDTH="80%"
ALIGN="center"
VALIGN="bottom"
>Chapter 3. Indexing</TD
><TD
WIDTH="10%"
ALIGN="right"
VALIGN="bottom"
><A
HREF="dpsearch-htmlparser.en.html"
ACCESSKEY="N"
>Next</A
></TD
></TR
></TABLE
><HR
ALIGN="LEFT"
WIDTH="100%"></DIV
><DIV
CLASS="sect1"
><H1
CLASS="sect1"
><A
NAME="stored"
>3.13. Storing compressed document copies</A
></H1
>
	
<A
NAME="AEN2295"
></A
>

	<P
>In <SPAN
CLASS="application"
>DataparkSearch</SPAN
> it is possible to store compressed copies of indexed
documents. Copies are stored and retrieved by the new daemon -
<B
CLASS="command"
>stored</B
>, that is installed into
<TT
CLASS="filename"
>sbin</TT
> directory of
<SPAN
CLASS="application"
>DataparkSearch</SPAN
> installation (default:
<TT
CLASS="filename"
>/usr/local/dpsearch/sbin</TT
>).</P
>

<P
><A
NAME="AEN2304"
></A
>
<A
NAME="AEN2307"
></A
>
<B
CLASS="command"
>stored</B
> is also used to make documents excerpts for search results.
<A
NAME="AEN2311"
></A
>
You may use <B
CLASS="command"
>ExcerptSize</B
> command in <TT
CLASS="filename"
>search.htm</TT
> template to specify average excerpt size in bytes.
Value by default: 256. You may switch off document excerpts (but retain ability to show stored copies) 
using <B
CLASS="command"
>DoExcerpt no</B
> command in your search template.
</P
>

<P
><A
NAME="AEN2318"
></A
>
To enable documents copies archiving without <B
CLASS="command"
>stored</B
> usage,
place <B
CLASS="command"
>DoStore yes</B
> command in your
<TT
CLASS="filename"
>indexer.conf</TT
> file instead of <B
CLASS="command"
>stored</B
> daemon configuration.
</P
>

	<P
>Stored document copies are retrieved by means of
<TT
CLASS="filename"
>storedoc.cgi</TT
> CGI script. It requests a saved copy
of a documents from <B
CLASS="command"
>stored</B
>, then a copy is
displayed with user's web browser with search keywords highlighted.
</P
>

<P
> To enable <B
CLASS="command"
>stored</B
> support, compile
<SPAN
CLASS="application"
>DataparkSearch</SPAN
> with <TT
CLASS="literal"
>zlib</TT
>
support:
	<PRE
CLASS="programlisting"
>&#13;./configure --with-zlib &#60;other arguments&#62;
</PRE
>
	</P
>


<P
>&#13;<A
NAME="AEN2334"
></A
>
<A
NAME="AEN2337"
></A
>
You may use the <B
CLASS="command"
>Store</B
> and <B
CLASS="command"
>NoStore</B
> commands to allow or disallow storing several files by pattern.
For arguments of those commands are exactly the same 
as for the <B
CLASS="command"
>Allow</B
> command (see ). All documents are stores by defaults, 
if support for stored is enabled.
</P
>


	<DIV
CLASS="sect2"
><H2
CLASS="sect2"
><A
NAME="stored-start"
>3.13.1. Configure stored</A
></H2
>
		
		<P
>To start using <B
CLASS="command"
>stored</B
>, please do the following:</P
>
		<P
></P
><UL
><LI
>&#13;<P
>Copy <TT
CLASS="filename"
>/usr/local/dpsearch/etc/stored.conf-dist</TT
> to
<TT
CLASS="filename"
>/usr/local/dpsearch/etc/stored.conf</TT
>.</P
>
				<P
>Edit <TT
CLASS="filename"
>/usr/local/dpsearch/etc/stored.conf</TT
>
				</P
>
				<P
>There are several commands available
for using with <B
CLASS="command"
>stored</B
>: </P
>
				<P
></P
><UL
><LI
>&#13;						<P
>&#13;<B
CLASS="command"
><A
NAME="AEN2361"
></A
>Listen</B
>
tells <B
CLASS="command"
>stored</B
> which address and/or port to bind
to. By default <B
CLASS="command"
>stored</B
> listens to port 7004 and any
address. It is possible to specify port only:</P
>
						<P
>&#13;							<KBD
CLASS="userinput"
>Listen 7004 </KBD
>
						</P
>
						<P
>Or address only:</P
>
						<P
>&#13;							<KBD
CLASS="userinput"
>Listen 127.0.0.2</KBD
>
						</P
>
						<P
>Or both address and port:</P
>
						<P
>&#13;							<KBD
CLASS="userinput"
>Listen 127.0.0.2:7004</KBD
>
						</P
>
					</LI
><LI
>&#13;						<P
>&#13;<B
CLASS="command"
><A
NAME="AEN2377"
></A
>VarDir</B
>
command specifies an alternate <TT
CLASS="filename"
>var/</TT
> working
directory, e.g.</P
>
						<P
>&#13;							<KBD
CLASS="userinput"
>VarDir /mnt/d/dpsearch/var/ </KBD
>
						</P
>
					</LI
><LI
>&#13;<P
><B
CLASS="command"
><A
NAME="AEN2386"
></A
>StoredFiles</B
>
command specifies number of stored data files created in <TT
CLASS="filename"
>var/stored/</TT
> directory,
e.g.</P
>
<P
>&#13;<KBD
CLASS="userinput"
>StoredFiles 256</KBD
>
</P
>
</LI
><LI
>&#13;<P
><B
CLASS="command"
><A
NAME="AEN2395"
></A
>OptimizeInterval</B
>
command specifies interval in seconds between attempts to optimize a stored datafile, e.g.</P
>
<P
>&#13;<KBD
CLASS="userinput"
>OptimizeInterval 300</KBD
>
</P
>
</LI
><LI
>&#13;<P
><B
CLASS="command"
><A
NAME="AEN2403"
></A
>OptimizeRatio</B
>
command  specifies defragmentation threshold for a stored datafile optimization.  
</P
>
<P
>&#13;<KBD
CLASS="userinput"
>OptimizeRatio 3</KBD
>
</P
>
</LI
></UL
>
			</LI
><LI
>&#13;				<P
>Run <B
CLASS="command"
>stored</B
>:</P
>
				<P
>&#13;					<KBD
CLASS="userinput"
>/usr/local/dpsearch/sbin/stored &#38; </KBD
>
				</P
>
			</LI
><LI
>&#13;				<P
>Configure <TT
CLASS="filename"
>indexer.conf</TT
> and <TT
CLASS="filename"
>search.htm</TT
>
(or <TT
CLASS="filename"
>searchd.conf</TT
>, if <B
CLASS="command"
>searchd</B
> is used). Specify address and port that <TT
CLASS="literal"
>indexer</TT
> will use to connect to <B
CLASS="command"
>stored</B
>. 
Use <TT
CLASS="literal"
>stored</TT
> parameter for <B
CLASS="command"
>DBAddr</B
> command, e.g.:</P
>
				<P
>&#13;					<KBD
CLASS="userinput"
>DBAddr mysql://localhost/search/?dbmode=cache&#38;stored=localhost:7004</KBD
>
				</P
>
			</LI
></UL
>
	</DIV
>
	<DIV
CLASS="sect2"
><H2
CLASS="sect2"
><A
NAME="stored-how"
>3.13.2. How stored works</A
></H2
>
		
		<P
>After you have successfully configured
<B
CLASS="command"
>stored</B
>, the <TT
CLASS="filename"
>indexer</TT
> pass
downloaded documents to <B
CLASS="command"
>stored</B
> daemon. After that,
<B
CLASS="command"
>stored</B
> will compress the received documents and
save them.</P
>
	</DIV
>
	<DIV
CLASS="sect2"
><H2
CLASS="sect2"
><A
NAME="stored-search"
>3.13.3. Using stored during search</A
></H2
>
		
		<P
>To enable displaying stored documents during search, do the following: </P
>
		<P
></P
><UL
><LI
>&#13;				<P
>Configure
<TT
CLASS="filename"
>storedoc.htm</TT
> (<TT
CLASS="filename"
>storedoc.cgi</TT
>
template)  if necessary.</P
>
			</LI
><LI
>&#13;				<P
>Add a <TT
CLASS="filename"
>storedoc.cgi</TT
> link to <TT
CLASS="filename"
>search.htm</TT
>
					<TT
CLASS="literal"
>&#60;!--res--&#62;</TT
> section: e.g. <TT
CLASS="literal"
>&#60;A HREF="$(stored_href)"&#62;Cached copy&#60;/A&#62; </TT
>
				</P
>
			</LI
><LI
>&#13;				<P
>Specify
<TT
CLASS="filename"
>storedoc.cgi</TT
> CGI script URL in
<TT
CLASS="filename"
>search.htm</TT
> (by default
<TT
CLASS="literal"
>$(stored_href)</TT
> will return
<TT
CLASS="filename"
>/cgi-bin/storedoc.cgi</TT
>). In case you have to
specify other URL, add to <TT
CLASS="filename"
>search.htm</TT
>
					<TT
CLASS="literal"
>variables</TT
> section the following line:</P
>
				<P
>&#13;					<KBD
CLASS="userinput"
>StoredocURL /path/to/storedoc.cgi</KBD
>
				</P
>
				<P
>Or an absolute path:</P
>
				<P
>&#13;					<KBD
CLASS="userinput"
>StoredocURL http://servername/path/to/storedoc.cgi</KBD
>
				</P
>
			</LI
></UL
>
		<P
>This is how <B
CLASS="command"
>stored</B
> works during search, if everything configured correctly:</P
>
		<P
></P
><OL
TYPE="1"
><LI
>&#13;				<P
>&#13;					<TT
CLASS="filename"
>search.htm</TT
> displays a link to <TT
CLASS="filename"
>storedoc.cgi</TT
>;</P
>
			</LI
><LI
>&#13;				<P
>When user clicks the link,
<TT
CLASS="filename"
>storedoc.cgi</TT
> sends a query to
<B
CLASS="command"
>stored</B
> daemon to the address, specified in
<TT
CLASS="filename"
>storedoc.htm</TT
> with the
<TT
CLASS="literal"
>Stored</TT
> parameter of <B
CLASS="command"
>DBAddr</B
> command;</P
>
			</LI
><LI
>&#13;				<P
>After the query, <B
CLASS="command"
>stored</B
> will decompress the requested saved copy and send it to <TT
CLASS="filename"
>storedoc.cgi</TT
>;</P
>
			</LI
><LI
>&#13;				<P
>&#13;					<TT
CLASS="filename"
>storedoc.cgi</TT
> parses the received document and apply highlighting of search keywords. Highlighting method is specified with <TT
CLASS="filename"
>storedoc.htm</TT
>
					<B
CLASS="command"
><A
NAME="AEN2482"
></A
>HlBeg</B
> and <B
CLASS="command"
><A
NAME="AEN2486"
></A
>HlEnd</B
> commands;</P
>
			</LI
></OL
>
	</DIV
>
</DIV
><DIV
CLASS="NAVFOOTER"
><HR
ALIGN="LEFT"
WIDTH="100%"><TABLE
SUMMARY="Footer navigation table"
WIDTH="100%"
BORDER="0"
CELLPADDING="0"
CELLSPACING="0"
><TR
><TD
WIDTH="33%"
ALIGN="left"
VALIGN="top"
><A
HREF="dpsearch-syslog.en.html"
ACCESSKEY="P"
>Prev</A
></TD
><TD
WIDTH="34%"
ALIGN="center"
VALIGN="top"
><A
HREF="index.en.html"
ACCESSKEY="H"
>Home</A
></TD
><TD
WIDTH="33%"
ALIGN="right"
VALIGN="top"
><A
HREF="dpsearch-htmlparser.en.html"
ACCESSKEY="N"
>Next</A
></TD
></TR
><TR
><TD
WIDTH="33%"
ALIGN="left"
VALIGN="top"
>Using syslog</TD
><TD
WIDTH="34%"
ALIGN="center"
VALIGN="top"
><A
HREF="dpsearch-indexing.en.html"
ACCESSKEY="U"
>Up</A
></TD
><TD
WIDTH="33%"
ALIGN="right"
VALIGN="top"
><SPAN
CLASS="application"
>DataparkSearch</SPAN
> HTML parser</TD
></TR
></TABLE
></DIV
><!--#include virtual="body-after.html"--></BODY
></HTML
>